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FINAL  REPORT 

Assessing  Student  Understanding  of  Concepts  in  Electricity 
to  inform  Instructional  Decisions 

ONR  Research  Group* 

Vanderbilt  University 
Box  1679,  Station  B 
Nashville,  TN  37235 


1.  INTRODUCTION 


We  have  been  investigating  students'  knowledge  and  understanding  of  ba¬ 
sic  concepts  in  electricity  and  their  application  to  solving  electrical  circuit  problems.  In 
previous  work,  we  identified  and  characterized  domain  concepts  that  students  had  diffi¬ 
culty  applying  correctly  to  problem  solving  tasks  mainly  in  the  DC  domain.  We  found 
that  student  knowledge  was  “in  pieces,”  and  their  lack  of  understanding  could  be  broadly 
classified  into  four  different  categories:  (i)  undifferentiated  concepts,  (ii)  experiential  im¬ 
poverishment,  i.e.,  the  inability  to  link  physical  processes  and  parameters  to  abstract  cir¬ 
cuit  models,  (iii)  incomplete  metaphors,  and  (iv)  simplifying  assumptions  of  minimum 
causality  [Biswas,  et  al.,  1997].  Moreover,  the  “invisible”  nature  of  electricity  made  it 
difficult  to  comprehend,  and  beginning  students  came  into  the  domain  with  very  few  pre¬ 
conceptions  (and,  therefore,  misconceptions).  Most  of  what  a  student  knew  was  picked 
up  from  instruction.  We  also  discovered  that  the  range  of  misconceptions  and  student 
learning  styles  were  best  handled  by  employing  different  perspectives  and  instructional 
resources.  We  developed  a  learning  environment  that  provided  resources  for  self- 
assessment  along  with  learning,  and  pilot  studies  showed  that  approach  to  be  quite  useful 
in  learning  difficult  concepts. 

In  this  phase  of  the  project,  our  initial  focus  was  on  characterizing  the  AC  circuit  do¬ 
main,  and  analyzing  student  understanding  and  problem  solving  ability  in  this  domain. 
Protocol  analyses  on  beginning  and  more  advanced  undergraduate  students  in  the  Electri¬ 
cal  Engineering  (EE)  program  revealed  that  students  have  very  little  physical  intuition  of 
AC  circuit  concepts.  Students’  problem  solving  primarily  involved  the  generation  of 
mathematical  formulations  (equations),  and  manipulating  these  formulations  to  derive 
numerical  solutions  to  problems.  Appeal  to  everyday  physical  phenomena  did  not  seem 
to  clarify  or  improve  the  students’  understanding  of  these  concepts.  For  example,  one  of 
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the  most  prevalent  misconceptions  among  beginning  EE  students  is  that  the  sinusoidal 
wavefonn  represents  a  spatial  property  of  the  voltage  and  current  in  a  wire  as  opposed  to 
a  time-vaiying  description  of  behavior  that  occurs  simultaneously  at  every  point  in  the 
wire.  These  students  also  have  no  notion  of  what  it  means  for  voltage  and  current  to  take 
on  negative  values.  Some  of  this  may  be  attributed  to  the  students’  lack  of  understanding 
of  the  physical  nature  of  voltage  and  current.  However,  a  more  direct  reason  may  be  the 
natural  mapping  those  students  create  from  the  visual  representation  of  the  sinusoidal 
wavefomi  to  the  spatial  dimension  of  a  wire.  Unlike  the  DC  domain,  our  attempts  to  link 
AC  waveforms  to  everyday  phenomena,  such  as  the  operation  of  radio  receivers,  and  the 
transmission  of  signals  from  different  radio  stations  to  a  receiver,  did  not  help  in  clarify¬ 
ing  misconceptions.  Students  also  had  a  lot  of  difficulty  in  understanding  the  behavior  of 
components,  such  as  capacitors  and  inductors,  which  exhibit  time  varying  behavior  in  AC 
circuits.  In  such  situations,  most  students  could  not  correctly  fomiulate  and  explain  the 
equations  for  analyzing  AC  circuit  behavior.  We  have  developed  a  test  in  the  AC  domain 
to  capture  the  primary  misconceptions  that  students  exhibit  in  understanding  AC  circuit 
behavior.  This  is  discussed  in  greater  detail  in  Section  3. 

Our  protocol  analyses  and  misconceptions  studies  have  established  that  students 
have  very  little  understanding  of  AC  circuit  concepts.  As  a  result,  they  exhibit  a  lot  of 
difficulty  in  formulating  and  solving  problems  in  this  domain.  Moreover,  a  number  of  the 
students’  misconceptions  and  difficulties  can  be  linked  to  instruction,  as  opposed  to  pre¬ 
conceived  notions  of  domain  concepts.  These  observations  have  led  us  to  turn  to  dynamic 
assessment  approaches  (Feurestein,  1979;  Campione  and  Brown,  1985  and  1987;  Brans- 
ford,  et  al,  1987)  and  focus  more  on  how  to  prepare  students  to  learn  through  instruction. 
This  methodology,  called  the  Assessment  for  Domain  Learnability  is  described  in  greater 
detail  in  Section  4. 

In  the  last  year,  we  have  decided  to  adopt  a  systematic  methodology  for  instruc¬ 
tion,  learning,  and  assessment  in  this  domain.  We  adopt  a  generic  framework  for  describ¬ 
ing  physical  systems  in  terms  of  their  structure,  behavior,  and  function.  We  link  this  de¬ 
scriptive  framework  to  three  broad  categories  of  problems  that  engineers  and  teclinicians 
encounter  in  their  everyday  work:  analysis,  diagnosis,  and  design.  These  three  tasks  can 
also  be  looked  upon  as  mappings  between  the  structure,  behavior,  and  function  of  a  cir¬ 
cuit,  and  the  fonnulation  is  used  to  develop  sets  of  questions  for  students  to  assess  their 
understanding  of  the  various  concepts  in  the  domain. 

To  aid  students  in  developing  a  systematic  and  well-structured  problem-solving 
paradigm,  we  have  adopted  an  instructional  strategy  that  emulates  expert  problem  solving 
behavior.  An  important  component  of  this  process  is  to  get  students  to  reason  about  phe¬ 
nomena  using  qualitative  techniques,  so  that  their  focus  is  on  the  application  of  the  laws 
that  govern  circuit  behavior,  and  not  on  mathematical  manipulations.  We  introduce  the 
notion  of  invariants  that  capture  the  fundamental  laws  and  concepts  that  govern  electrical 
circuit  behavior.  We  have  developed  a  web-based  software  system  for  self-assessment 
and  learning  called  Inductor  (described  in  Section  5)  that  presents  students  with  a  set  of 
multiple-choice  questions  about  a  variety  of  AC  circuits.  The  sequence  goes  from  simpler 
questions  to  progressively  more  difficult  ones,  and  starts  from  purely  resistive  circuits, 
and  then  goes  onto  RC  and  RLC  circuits  linked  to  different  real-world  applications.  Stu- 
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dents  are  required  to  pick  the  relevant  invariant  relations  and  analyze  them  qualitatively 
to  derive  the  solution  to  the  problem.  A  detailed  description  of  the  system,  and  prelimi¬ 
nary  experiments  that  we  have  conducted  with  the  system  are  described  in  Section  5. 

The  report  ends  with  a  summary  of  the  current  status  of  our  work,  and  proposes  direc¬ 
tions  for  future  research.  A  set  of  appendices  provides  details  of  the  Inductor  system. 

2.  AC  DOMAIN  DESCRIPTION 


Like  DC  circuits,  the  fundamentals  of  the  AC  domain  are  represented  in  terms  of 
voltage,  current,  and  power.  In  AC  circuits,  these  values  are  time  varying,  and  described 
visually  as  waveforms,  most  typically  sinusoidal  waveforms.  Two  parameters  of  these 
waveforms,  the  frequency  and  the  phase,  play  an  important  role  in  characterizing  the  be¬ 
havior  of  AC  circuits.  Typically  beginning  students  are  able  to  reproduce  voltage  and  cur¬ 
rent  values  in  mathematical  (the  sinusoidal  equation)  and  visual  fonns  (sine  waves),  but 
do  not  really  understand  the  link  between  the  waveforms  and  the  voltage  drops  and  cur¬ 
rent  flows  in  a  given  circuit. 

The  time-varying  nature  of  voltage  and  current  is  the  basis  for  the  differences  in 
AC  and  DC  circuit  analysis.  For  purely  resistive  circuits,  this  difference  is  not  significant 
because  voltage  and  current  remain  in  phase,  and  resistance  values  are  not  affected  by 
frequency  changes.  Therefore,  voltage  and  current  computations  are  based  on  simple  al¬ 
gebraic  relations.  Power  computations  in  AC  circuits  have  an  equivalent  DC  expression 
when  voltages  and  currents  are  expressed  as  root  mean  square  (RMS)  values. 

Capacitor  and  inductor  elements  exhibit  significantly  different  behaviors  in  AC 
circuits.  Their  impedance  values  (the  equivalent  of  resistance)  are  a  function  of  the  fre¬ 
quency  of  the  AC  waveform,  and  this  property  is  exploited  in  the  design  of  a  number  of 
applications.  Capacitor  and  inductor  elements  also  cause  a  phase  difference  between  volt¬ 
age  and  current,  and  this  is  used  in  the  design  of  applications  like  filters,  oscillators,  and 
signal  generators. 

Our  approach  to  analyzing  student  understanding  of  DC  and  AC  concepts  is  based 
on  the  observation  that  the  two  domains  share  a  number  of  fundamental  concepts.  Our 
protocol  studies  on  AC  understanding  were  divided  into  two  phases.  The  first  phase  fo¬ 
cused  on  these  basic  concepts.  The  second  phase  looked  at  more  advanced  AC  concepts 
in  the  context  of  applications.  The  primary  applications  of  AC  systems  are  in  power 
transmission,  broadcasting,  and  communication.  AC  is  still  the  most  effective  way  for 
power  generation  and  transmission,  but  in  the  present  day  digital  generation,  most 
equipment,  such  as  computers,  convert  the  input  AC  voltage  to  a  DC  voltage  before  use. 
Communication  systems  use  AC  waveforms  superimposed  on  DC  signals  for  their  opera¬ 
tion.  In  keeping  with  our  previous  protocol  studies  (Biswas  et  al.,  1997;  Schwartz, 
Biswas,  Bransford,  Bhuva,  Balac,  &  Brophy,  2000),  where  we  studied  DC  concepts  in 
the  context  of  real-world  devices,  our  study  of  student  understanding  of  advanced  AC 
concepts  has  been  in  the  context  of  the  applications  discussed  above. 
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3.  MISCONCEPTION  STUDIES 


We  briefly  review  previous  work  in  analyzing  misconceptions  in  the  domain  of  elec¬ 
tricity.  Most  of  this  work  has  been  targeted  to  DC  circuits.  We  extend  the  analysis  of  DC 
misconceptions  to  the  AC  circuit  domain,  and  present  the  results  of  our  protocol  studies. 
To  analyze  misconceptions  in  a  more  systematic  way,  we  have  developed  a  misconcep¬ 
tions  test  for  AC  concepts.  We  briefly  describe  the  test  in  this  report.  The  complete  set  of 
test  question  can  be  accessed  at  http://relax.ltc.vanderbilt.edu/onr/ac-misconceptions.doc. 

Previous  Work 

The  DC  misconception  literature  lists  the  erroneous  conceptions  students  have 
about  the  domain  as  well  as  the  omissions  of  knowledge  that  they  demonstrate.  In  our 
previous  work  (Biswas,  et  al,  1997;  Schwartz,  et  al,  2000)  we  categorize  and  report  most 
of  the  known  misconceptions  and  omissions  that  students  have  about  the  notion  of  volt¬ 
age,  current,  resistance,  power  and  other  electrical  circuit  concepts. 

Cohen,  Eylon,  and  Ganiel  (1982)  found  that  students  think  of  current  as  the  pri¬ 
mary  concept  (potential  difference  is  regarded  as  a  consequence  of  current  flow,  and  not 
as  its  cause),  and  that  the  battery  is  often  regarded  as  a  source  of  constant  current  rather 
than  constant  voltage.  They  also  observed  students'  "difficulties  in  analyzing  the  effect 
that  a  change  in  one  component  has  on  the  rest  of  the  circuit"  and  dealing  with  a  simulta¬ 
neous  change  of  several  variables.  These  misconceptions  cause  major  problems  in  stu¬ 
dents'  reasoning  about  electrical  circuits.  Other  literature  in  the  field  concentrated  on 
student  understanding  using  analogical  models.  For  example.  Centner  and  Centner 
(1993)  dealt  with  two  different  analogical  models;  (i)  the  “flowing  water -model,”  where 
the  flow  of  current  through  wires  is  analogical  to  the  flow  of  water  through  pipes  and  (ii) 
the  “teeming  crowds  model,”  where  the  analogy  was  made  between  current  or  the  flow  of 
charged  particles  and  the  movement  of  crowds  through  passageways.  Magnusson,  Tem¬ 
ple,  and  Boyle  (1997)  discovered  eight  different  students'  models  of  the  path  of  electric 
cun-ent  in  parallel  circuits  and  adapted  six  different  models  of  students'  conceptions  of 
cuirent  from  work  reported  in  Osborne  (1983),  Russell  (1980),  and  Arnold  and  Millar 
(1987). 


Hunt  &  Minstrell  (1994)  have  generated  a  list  of  pre-scientific  knowledge  pieces, 
or  facets,  that  students  may  have,  including  misconceptions  about  concepts  in  electricity. 
They  developed  a  program  (DIACNOSER)  that  targets  and  assesses  these  misconcep¬ 
tions  with  carefully  constructed  test  questions.  Upon  identifying  a  specific  difficulty  a 
student  has,  DIACNOSER  also  provides  some  instruction  and  resources  addressing  this 
misconception. 

AC  Misconception  Studies 

We  have  begun  extending  the  work  on  student  understanding  and  misconceptions 
in  the  DC  circuit  domain  to  the  AC  and  DC  domains.  We  generated  a  series  of  circuit 
questions  relevant  to  the  AC  domain  and  interviewed  students  as  they  worked  through 
these  problems.  As  a  result  of  these  structured  interviews,  we  identified  specific  areas  in 
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which  students  had  misconceptions  or  lacked  experience  (listed  later  in  this  section). 
More  recently  we  also  constructed  part  of  a  misconceptions  multiple  choice  test  that  tar¬ 
gets  these  misconceptions,  in  cooperation  with  Steve  Parchman  and  other  researchers 
(also  described  later  in  this  section). 

Experimental  Setup 

For  the  protocol  analysis  studies,  we  made  up  a  number  of  AC  problems,  starting 
from  the  simple  DC  flashlight,  but  replacing  the  DC  source  with  an  AC  source.  The  first 
set  of  problems  were  set  up  for  students  to  analyze  contrasting  cases,  such  as  what  hap¬ 
pens  in  the  flashlight  circuit  when  the  DC  souree  is  replaced  by  an  AC  source,  and  where 
would  you  place  fuses  to  protect  a  component  in  identical  DC  and  AC  circuits.  In  this 
study,  we  were  specifically  looking  for  misconceptions  that  students  had  exhibited  in  an 
earlier  study  on  DC  circuits,  such  as  (i)  the  empty  pipe  and  sequential  flow  misconcep¬ 
tions,  (ii)  the  inability  to  recognize  the  differences  between  voltage  and  current,  and  (iii) 
the  belief  that  current  remained  constant  in  a  circuit,  and  what  impact  these  misconcep¬ 
tions  may  have  on  their  understanding  of  AC  circuits.  In  addition,  there  were  questions 
that  asked  students  to  analyze  the  effect  of  changing  source  frequency  on  power  con¬ 
sumed  in  a  circuit.  In  some  cases,  the  students  were  asked  to  plot  the  voltage  and  current 
waveforms  at  different  points  in  a  circuit.  The  students  involved  in  this  study  were  begin¬ 
ning  Electrical  Engineering  (EE)  students  at  Vanderbilt  University  who  had  completed 
their  first  circuits  course.  We  also  interviewed  students  in  the  Navy  training  center  at 
Memphis. 

We  also  developed  a  second,  more  advanced  AC  problem  set,  where  students 
were  asked  to  explain  how  a  particular  device  worked,  and  especially  why  it  exhibited 
certain  behaviors  and  functionality.  The  second  set  of  problems  tested  student  under¬ 
standing  of  capacitors  and  inductors  in  AC  circuits,  and  the  use  of  RC  and  RLC  circuits 
in  practical  applications.  This  set  of  problems  was  presented  to  senior  undergraduate  stu¬ 
dents  and  some  graduate  students.  We  also  interviewed  an  electrical  technician.  The  fo¬ 
cus  was  on  whether  students  could  analyze  the  circuits  and  produce  a  qualitative  explana¬ 
tion  of  the  observed  system  functionality.  Our  last  report  [Biswas,  et  al.,  1999]  describes 
the  problem  sets  in  greater  detail. 

AC  Misconceptions 

The  analysis  of  student  responses  provided  interesting  results.  We  interviewed  a 
total  of  18  subjects  at  Vanderbilt  University,  and  about  6  trainees  in  their  first  EE  techni¬ 
cian  course  at  the  Memphis  naval  center.  All  12  Vanderbilt  students  in  the  first  group 
were  in  the  beginning  electrical  engineering  course  (EECE  112),  and  the  6  students  in  the 
second  group  were  juniors,  seniors,  and  graduate  students.  In  our  protocol  analysis  we 
found  a  variety  of  erroneous  knowledge  about  basic  AC  concepts.  They  are  summarized 
below.  We  divided  the  misconceptions  into  three  categories: 

1 .  Those  directly  related  to  characteristics  of  AC  waveforms, 

2.  General  classes  of  difficulties  that  are  linked  to  cognitive  difficulties,  and 

3.  Lack  of  knowledge  of  general  domain  principles. 
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These  are  discussed  in  greater  detail  below. 


List  of  Misconceptions  specific  to  AC  waveforms. 

1.  Spatial  AC  misconception.  The  sinusoidal  AC  voltage  and  current  waveforms  are  not  a 
representation  of  variation  of  these  variables  at  a  point  in  time.  Rather  they  depict  a  varia¬ 
tion  of  their  magnitudes  along  the  length  of  the  wire  in  which  the  cuixent  is  flowing.  For 
example,  students  said  that  a  string  of  identical  light  bulbs  in  series  when  connected  to  an 
AC  source  would  light  up  in  sequence,  and  some  of  the  light  bulbs  may  be  on  when  others 
are  off.  At  the  same  instant  of  time,  the  brightness  of  the  bulbs  would  vary  depending  on 
their  position  in  the  circuit. 

2.  Negative  part  of  AC  cycle  is  just  a  mathematical  artifact.  No  current  flowing  in  circuit 
or  power  delivered  during  negative  part  of  AC  cycle.  For  example,  a  number  of  students 
said  that  a  light  bulb  only  lights  up  during  the  positive  part  of  the  sinusoidal  cycle.  Others 
said  that  there  could  be  '*no  such  thing  as  negative  current.  That  is  just  a  mathematical  ar¬ 
tifact.  If  current  reverses,  the  electrons  would  reverse  direction  too.  They  would  then  run 
into  each  other,  stopping  flow,  which  implies  there  could  be  no  current.  ” 

3.  Alternate  form  of  this  misconception.  The  negative  current  ’’cancels”  out  the  positive 
current.  So  bulb  will  never  light  up  when  you  connect  to  true  AC  source. 

4.  Empty  pipe  misconception.  During  AC  cycle  electrons  stop,  turn  around,  and  go  the 
other  way.  In  some  cases  when  you  have  very  long  wires,  they  may  never  reach  the  light 
bulb  connected  to  the  end  of  the  wire.  Students  thought  that  you  would  need  two  fuses  to 
provide  protection  in  an  AC  circuit,  where  you  could  do  with  one  in  a  DC  circuit. 

5.  Incorrectly  importing  DC  models  to  explain  AC. 

A.  Students  often  surmised  that  the  alternating  current  going  through  a  resistor  was 
constant  in  time. 

B.  Students  often  hypothesized  that  a  capacitor  behaved  the  same  in  AC  and  DC 
circuits. 

6.  Difficulties  understanding  circuit  behavior  when  AC  and  DC  signals  are  combined. 

Students  had  difficulty  “separating”  or  recognizing  the  AC  and  DC  components  of  a  sig¬ 
nal  in  problems  in  which  the  midpoint  of  a  sinusoidal  voltage  was  not  zero. 

7.  More  generally,  difficulty  thinking  of  circuit  behavior  when  multiple  waveforms, 
frequencies  are  combined.  Even  advanced  students  stated  that  the  number  of  channels 
you  can  got  from  cable  TV  was  a  function  of  the  number  of  wires  in  the  cable,  or  the 
thickness  of  the  cable. 

General  classes  of  difficulties  that  are  not  specific  to  AC  [Schwartz,  et  al.  2000] 

8.  Failure  to  differentiate  among  concepts.  Examples,  voltage  and  current,  series  and  par¬ 
allel  configurations,  role  of  capacitor  in  DC  versus  AC  circuits. 
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9.  Minimum  causality  error.  (Incorrect  simplifying  assumptions).  Single  change  in  out¬ 
come  must  be  a  result  of  single  change  in  cause,  (e.g.,  a  lOW  bulb  must  have  greater  resis¬ 
tance  than  a  5W  bulb). 

10.  Overly  local  reasoning.  Not  thinking  of  global  constraints,  invariants. 

1 1 .  Bad  framing.  Incorrect  generalizations,  trouble  switching  from  equations  to  physical 
explanations  to  analogical  models. 

12.  Experiential  impoverishment.  Electricity  is  invisible  except  for  its  end  products. 

Lack  of  basic  circuit  knowledge. 

13.  Lack  of  Ohm's  law  (how  resistance  affects  current  when  voltage  is  constant) 

1 4.  Lack  of  KCL  (current  through  all  components  of  a  loop  must  be  equal). 

15.  Lack  of  KVL  (the  voltage  drop  across  components  of  a  loop  must  sum  to  zero). 

Note  that  14  and  15  together  represent  the  consei'vation  laws:  (i)  charge  cannot  disappear, 

and  (ii)  energy  must  be  conserved. 

16.  Lack  of  knowledge  of  the  behavior  of  capacitors  (such  as  C=Q/V) 

17.  Lack  of  knowledge  of  Capacitor  and  Inductor  impedance  as  a  function  of  frequency. 

18.  Topographic  misunderstanding  of  the  circuit  (e.g.  unable  to  differentiate  series  from 
parallel). 


Misconceptions  Test 

Using  the  above  list  of  AC  misconceptions,  we  developed  a  set  a  number  of  mul¬ 
tiple  choice  test  questions  to  target  these  misconceptions  in  cooperation  with  Steve 
Parcliman’s  group  in  Florida,  and  other  researchers.  An  example  question  is  shown  be¬ 
low  in  Figure  1 . 

This  question  focuses  on  the  spatial  misconception  that  students  have  regarding 
electricity  (water-pipe  model,  and  the  spatial  variation  of  AC  signals).  The  question  also 
addresses  the  notion  of  electricity  as  a  substance,  i.e.,  electricity  gets  consumed  as  it  goes 
along  the  string  of  lights. 

The  misconceptions  test  has  been  conducted  with  naval  students,  and  Steven 
Parclmian’s  group  is  currently  analyzing  the  results.  The  full  test  can  retrieved  from 
http://relax.ltc.vanderbilt.edu/onr/ac-misconceptions.doc. 

Discussion 

Our  preliminary  study  of  student  understanding  in  the  AC  domain  has  proven  to 
be  quite  revealing.  Beginning  students  seem  to  have  very  little  understanding  of  the 
time-varying  nature  of  AC  voltage  and  current.  This  can  be  attributed  to  a  combination  of 
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problems  they  exhibit  in  their  basic  understanding  of  concepts.  The  empty  pipe  miscon¬ 
ception  affects  their  understanding  of  current  flow,  and  makes  it  especially  difficult  for 
them  to  reason  about  current  that  reverses  direction  periodically.  The  inability  to  differen¬ 
tiate  between  voltage  and  current  and  the  lack  of  understanding  in  mapping  from  physical 
concepts  to  abstract  circuit  parameters  compounds  students’  problems.  They  are  often 
stuck  with  beliefs  such  as  a  source  provides  constant  current,  and  a  source  cannot  deliver 
power  unless  the  current  flows  in  one  direction  from  one  of  its  terminals  to  another. 
These  misconceptions  and  lack  of  knowledge  are  not  unique  to  the  AC  domain;  in  fact 
students  exhibited  the  same  problems  when  reasoning  in  the  DC  domain. 


50  3_lbs 

M - !► 


A  Cluistmas  light  strand  contains  50  identical  light  bulbs  connected  in  series  to  form  a  light  string.  When  it 
is  plugged  into  a  1 10  volt  AC  power  socket  of  frequency  60Hz,  which  light  will  bum  the  brightest? 

a)  The  first  bulb  is  always  the  brightest. 

b)  The  50*''  bulb  always  bums  the  brightest. 

c)  Since  the  current  is  alternating,  each  of  the  bulbs  starting  from  the  first  to  the  50"'  is  the 
brightest  in  turn. 

d)  All  of  the  bulbs  are  equally  bright  at  all  times. 

Conect  answer: 

d)  All  of  the  bulbs  are  equally  bright  at  all  times. 

Figure  1:  A  Misconceptions  Test  question  and  accompanying  figure 


From  the  point  of  view  of  instruction,  these  observations  can  be  interpreted  in 
many  ways.  On  the  one  hand,  one  can  make  the  argument  that  since  DC  instruction  tradi¬ 
tionally  precedes  AC  instruction,  it  is  very  important  to  ensure  that  students  do  not  de¬ 
velop  misconceptions  and  omissions  described  above  during  DC  instruction.  Careful  con¬ 
trasts  also  need  to  be  made  when  making  the  transition  from  the  DC  to  the  AC  domain. 
On  the  other  hand,  one  could  say  that  the  similarity  of  the  basic  concepts  in  the  two  do¬ 
mains  imply  that  the  most  effective  form  of  teaching  should  focus  on  the  concepts  and 
their  implications  in  problem  solving  rather  than  spend  a  lot  of  effort  in  focusing  on  the 
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differences.  For  resistive  circuits,  the  time-varying  nature  of  AC  voltage  and  current  has 
no  strong  implications  on  behavior.  Students  need  to  understand  the  concept  of  power 
delivered,  and  how  to  compute  the  power  delivered.  As  discussed  earlier,  the  time- 
varying  nature  of  current  and  voltage  has  important  implications  in  circuits  with  capaci¬ 
tors  and  inductors,  and  it  may  be  best  to  introduce  these  concepts  by  demonstrating  their 
use  in  real  applications  and  devices.  The  latter  approach  may  be  further  justified  by  the 
obseiwation  that  a  number  of  the  misconceptions  of  the  beginning  students  seemed  to  go 
away  as  they  moved  on  to  more  advanced  courses. 

Another  issue  of  importance  that  we  have  observed  among  students  is  their  reli¬ 
ance  on  mathematical  formulations  and  solving  of  equations  to  derive  answers  to  prob¬ 
lems.  As  discussed  earlier,  this  implies  the  students  lack  understanding  of  the  underlying 
physical  phenomena,  and  therefore,  do  not  develop  a  deep  understanding  of  the  basic 
concepts  in  the  domain.  This  problem  is  even  further  compounded  in  the  AC  domain, 
especially  when  students  have  to  deal  with  the  more  complex  phenomena  associated  with 
real  world  devices  and  systems.  When  dealing  with  the  questions  in  problem  set  2,  a 
number  of  students  attempted  to  convert  the  given  circuit  or  problem  description  into 
mathematical  equations.  However,  the  resultant  differential  equations  were  hard  to  ana¬ 
lyze,  and  did  not  directly  provide  the  information  required  to  solve  the  problem.  The  im¬ 
plication  here  is  that  students  need  to  develop  a  better  qualitative  understanding  of  phe¬ 
nomena,  and  how  these  phenomena  combine  to  produce  circuit  and  system  functionality. 
In  our  protocol  studies  on  the  second  problem  set,  a  number  of  students  had  to  be 
coached  to  reason  about  a  problem  qualitatively.  Only  then  were  they  able  to  analyze  the 
problem,  and  generate  the  desired  solutions  and  explanations.  Developing  qualitative  rea¬ 
soning  skills  and  function-level  understanding  may  also  contribute  to  the  development  of 
better  troubleshooting  skills,  a  long-term  goal  of  this  research. 

In  the  next  section  of  the  report,  we  develop  a  methodology  for  instruction  that 
combines  learning  with  assessment.  The  goal  is  to  exploit  computer  technology  to  pro¬ 
vide  students  with  an  environment  for  selecting  from  a  set  of  available  resources  depend¬ 
ing  on  their  self-identified  needs. 


4.  FROM  PROTOCOL  ANALYSIS  TO  INSTRUCTION: 

The  Assessment  of  Domain  Learnability  Framework 

Our  studies  of  student  understanding  in  AC  and  DC  circuit  problem  solving  sug¬ 
gested  that  student  misconceptions  and  difficulties  could  be  linked  to  instruction  as  op¬ 
posed  to  the  preconceived  notions  of  domain  concepts.  These  observations  led  us  to  turn 
to  dynamic  assessment  approaches  (Feurestein,  1979;  Campione  and  Brown,  1985  and 
1987;  Bransford,  et  al,  1987)  and  focus  more  on  how  to  prepare  students  to  learn  tlu'ough 
instruction.  Our  first  step  in  this  direction  was  to  build  a  computer-based  tools  using  the 
STAR.Legacy  framework  to  help  students  self-assess  their  understanding  of  concepts 
linked  to  DC  circuit  problem  solving,  and  to  provide  resources  to  help  students  learn 
these  concepts  they  found  difficult  to  learn. 
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Assessing  Domain  Learnability 

It  appears  that  some  electricity  concepts  may  be  more  difficult  to  learn  than  others. 
With  respect  to  the  instruction  in  this  domain,  we  believe  that  an  important  research  task 
is  to  identify  features  and  concepts  that  influence  learnability  of  concepts  that  affect  prob¬ 
lem  solving  tasks.  We  will  call  this  task  "assessing  domain  learnability”  or  ADL  for 
short.  By  trying  to  remediate  people's  misconceptions  and  missing  conceptions,  we  may 
detenuine  which  are  particularly  difficult  to  remediate  given  our  methods  of  instruction 
(e.g.,  Heller  &  Finley,  1992),  and  which  type  of  understanding  has  the  greatest  impact  on 
subsequent  learning.  The  basic  observation  is  that  not  all  misconceptions  are  equally 
strong  or  equally  relevant  to  future  instruction.  For  example,  although  we  have  rarely 
seen  it  in  the  literature  (Cooke  &  Breedin,  1 994),  it  would  be  interesting  to  ask  people  to 
compare  their  confidence  in  answers  where  they  exhibit  misconceptions  relative  to  those 
that  they  do  not.  We  suspect  that  for  many  of  the  misconceptions  that  have  been  docu¬ 
mented,  people  are  reasonably  aware  that  they  do  not  know  what  they  are  talking  about. 
For  those  misconceptions  that  are  of  low  confidence,  should  we  expect  that  people  would 
be  more  likely  to  overcome  their  misconceptions  and  learn?  Much  of  the  research  on 
misconceptions  has  no  handle  on  this  question.  An  ADL  approach  seems  more  likely  to 
provide  an  answer. 

There  may  be  limitations  to  ADL  as  we  have  conceptualized  it  so  far.  One  possi¬ 
ble  weakness  of  ADL  is  that  it  is  particularly  prone  to  the  ways  that  we  assess  whether 
someone  has  learned  a  correct  conception  or  not.  For  example,  if  we  ask  the  exact  same 
question  that  we  taught,  does  this  mean  that  people  have  learned  in  any  meaningful 
sense?  The  problem  of  assessing  and  deciding  upon  ecologically  satisfactory  understand¬ 
ing,  however,  is  a  problem  faced  by  much  educational  research.  ADL  actually  fairs  bet¬ 
ter  than  most  in  this  regard.  This  is  because  the  ultimate  test  for  ADL  is  whether  a  given 
concept  has  implications  for  future  learning.  For  example,  consider  the  typical  course 
sequence  in  electrical  engineering  where  students  begin  with  direct  current  (DC)  circuits 
and  then  move  to  study  alternating  current  (AC)  circuits.  Students  start  with  many  mis¬ 
conceptions  about  DC  circuits.  Are  all  the  misconceptions  and  their  correct  counterparts 
equally  important  in  shaping  students'  ability  to  learn  AC  circuits?  This  is  the  question 
that  ADL  is  designed  to  answer. 

A  second  potential  weakness  to  ADL  is  that  if  our  instruction  fails  to  teach  a  cor¬ 
rect  conception  of  a  domain,  it  is  hard  to  determine  whether  this  was  a  function  of  the 
domain's  difficulty  or  a  function  of  our  teaching  methods.  On  the  one  hand,  we  can  never 
disentangle  these  two  possibilities  beyond  a  reasonable  appraisal.  On  the  other  hand,  it  is 
the  interactions  of  the  instruction  and  the  domain  that  constitute  the  important  parameters 
of  assessing  domain  learnability.  The  emphasis  of  ADL  is  not  on  domain  learnability  in 
the  abstract,  but  rather  domain  learnability  with  respect  to  the  state  of  the  art  in  instruc¬ 
tion.  The  next  section  describes  a  computer  environment  that  captures  many  of  our  ideas 
about  the  state  of  the  art. 

A  Computer-Based  Learning  Environment  for  DC  Problem  Solving 

A  computer-based  environment  provides  an  integrated  learning-assessment  tool  for  pull¬ 
ing  together  different  instructional  techniques  and  resources  that  can  be  applied  to  a  do- 
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main.  A  single  instructional  technique  would  be  too  restrictive  for  ADL.  For  example, 
one  might  use  a  dynamic  tutoring  system  to  teach  the  procedural  knowledge  of  a  domain, 
but  there  are  other  types  of  knowledge  that  are  important  to  assess  as  well,  like,  do  people 
have  difficulty  constructing  a  mental  model  of  the  domain  (Lajoie  &  Lesgold,  1992). 
Similarly,  one  might  create  a  system  that  matches  an  individual's  misconceptions  against 
a  known  "bug  list"  and  teaches  to  those  bugs  directly,  but  this  typically  assumes  that  mis¬ 
conceptions  are  non-interacting. 

Our  software  environment  for  assessing  the  leamability  of  DC  concepts  was  cre¬ 
ated  using  the  STAR.Legacy  framework  (Schwartz,  et  al.  2000).  In  line  with  the  test- 
teach-retest  model  of  dynamic  assessment,  students  begin  with  a  question  in  the  look 
ahead  problem  and  end  with  the  same  question  when  they  reflect  back.  In  this  case,  the 
look  ahead  and  reflect  back  problem  asks  students  to  explain  what  happens  in  a  simple 
flashlight  circuit  when  a  5-watt  bulb  is  replaced  by  a  10-watt  bulb.  The  overall  assess¬ 
ment  and  learning  task  is  divided  into  three  challenges,  whieh  were  chosen  on  the  basis 
of  our  protocol  research  described  earlier  (Biswas,  et  al,  1997).  We  found  three  problem 
situations  that  were  particularly  good  at  making  students'  thinking  visible.  Challenge  1 
asked  students  to  reason  about  the  possible  causes  of  a  dim  bulb.  This  problem  was  in¬ 
tended  to  help  students  differentiate  voltage  and  current,  to  help  them  overcome  the 
minimum  causality  error,  and  to  give  them  some  increased  experience  in  the  domain  and 
its  analogies.  Challenge  2  asked  students  to  design  a  battery  operated  drill  that  could  run 
at  different  speeds.  In  this  design  problem,  students  progressively  deepen  their  under¬ 
standing  of  the  topics  raised  by  challenge  1  while  adding  the  issues  of  local  reasoning 
and  framing.  Finally,  challenge  3  tried  to  bring  the  lessons  together  into  a  single  prob¬ 
lem.  In  this  challenge,  students  were  asked  to  reason  about  a  flashlight  that  has  two 
bulbs,  one  that  points  forward  and  one  that  points  to  the  ground.  They  are  told  that 
somebody  wants  to  change  the  forward  bulb  to  a  higher  wattage.  How  will  that  effect  the 
flashlight  overall?  These  challenges  are  intended  to  bring  forward  the  different  classes  of 
misconceptions  that  students  may  possess.  At  the  same  time,  we  expect  the  interaction  of 
the  challenges  and  instruction  to  reveal  other  conceptual  hot  spots.  This  is  one  of  the  at¬ 
tractive  features  of  ADL  —  it  can  reveal  misconceptions  in  the  context  of  instruction. 

After  reading  a  challenge,  students  try  to  generate  their  first  thoughts  about  how 
to  prepare  for  solving  the  challenge.  These  initial  thoughts  usually  provide  both  instruc¬ 
tors  and  the  student  with  a  sense  of  the  strengths  and  weaknesses  of  the  student,  and  it 
helps  the  student  choose  which  of  the  multiple  perspectives  to  listen  to.  Each  perspective 
directly  targets  key  learning  difficulties  with  a  10-15  second  comment  by  an  expert.  For 
example,  one  of  the  perspectives  has  an  expert  explain  the  minimum  causality  error,  al¬ 
though  not  in  those  terms.  The  expert  states,  "a  common  mistake  that  people  make  with 
these  problems  is  that  they  often  do  not  realize  that  when  the  power  changes,  two  other 
things  in  the  circuit  must  change.  ”  Another  perspective  tries  to  tie  the  perceptual  phe¬ 
nomena  (a  dim  bulb)  to  relevant  electrical  concepts  by  pointing  out  that  a  dim  bulb  means 
less  power  is  being  consumed.  Another  perspective,  under  the  assumption  that  the  stu¬ 
dents  have  been  taught  some  fonn  of  water  analogy,  tries  to  get  students  to  think  how 
voltage  and  current  map  into  the  water  domain. 
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When  students  listen  to  the  perspectives,  instructors  may  ask  the  student  to  ex¬ 
plain  whether  they  understand  what  the  experts  are  saying.  This  provides  them  with 
valuable  knowledge  about  which  aspects  of  the  domain  the  student  may  be  having  trouble 
with.  For  example,  some  students  do  not  know  that  "two  things  must  change,  ”  whereas 
others  may  not  know  how  to  draw  the  analogy  between  water  and  electricity.  This  be¬ 
comes  important  when  the  interview  proceeds  to  Research  &  Revise.  The  student  and 
interviewer  choose  which  resources  to  work  with  depending  on  the  gaps  in  knowledge. 

Figure  2  shows  the  resources  that  are  available  for  challenge  1 .  A  chalk  talk  on 
Ohm's  law  explains  why  two  things  must  change  if  the  power  changes.  There  is  also  a 
set  of  multiple-choice  problems  that  allow  students  to  practice  using  Ohm's  law.  These 
problems  include  automated  feedback  that  states  the  qualitative  implications  of  the  stu¬ 
dent's  incorrect  answers.  For  example,  one  feedback  comment  reads,  "This  answer  im¬ 
plies  that  as  you  increase  the  voltage  across  the  circuit,  current  will  decrease!  For  exam¬ 
ple,  if  we  used  a  more  powerful  battery,  the  current  in  the  flashlight  circuit  would  de¬ 
crease.  Does  that  make  sense?"  This  form  of  feedback  helps  the  students  to  think  about 
qualitative  relationships  as  opposed  to  simply  making  algebraic  manipulations  of  num¬ 
bers. 


Figure  2:  Resources  for  challenge  1:  by  clicking  on  an  image,  a  learner  can  gain  access  to  its 
resources 

Another  resource  is  a  brief  presentation  of  a  mnemonic  that  helps  students  memo¬ 
rize  that  current  is  a  "tlirough"  property  whereas  voltage  is  an  "across"  property.  There 
are  also  pairings  of  simulations  of  a  circuit  and  an  analogous  water  system.  The  resource 
page  also  includes  connections  to  web  sites  that  we  have  found  helpful,  comments  by 
students  who  have  completed  the  process  and  offer  their  thoughts  about  key  insights  that 
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helped  their  learning,  and  pointers  to  simulations  and  hands-on  activities  developed  by 
others  (e.g.,  Parchman,  1997). 

Depending  on  how  comfortable  and  confident  students  feel  about  the  material, 
they  can  move  between  resources  and  perspectives  to  probe  further  and  learn  more  about, 
the  relevant  concepts.  Once  the  students  feel  that  they  have  made  satisfactoiy  learning 
progress,  they  move  to  test  your  mettle  to  test  the  strength  of  their  knowledge.  After  stu¬ 
dents  complete  the  learning  cycle  for  challenge  1,  they  move  to  subsequent  challenges, 
which  require  students  to  rethink  concepts  that  they  have  already  learnt,  and  also  to  deal 
with  new  concepts  and  misconceptions.  Subsequent  challenges  are  structured  like  the  first 
challenge. 

This  dynamic  assessment  environment  is  different  from  other  dynamic  assessment 
models  that  are  automated  (Lajoie  &  Lesgold,  1992)  because  it  keeps  the  instructor  in  the 
loop.  In  part  this  is  because  it  makes  it  much  easier  for  others  to  replicate  our  efforts  as 
compared  to  the  overhead  of  creating  automated  or  self-contained  systems  (Bell,  1998; 
Murray,  1998).  But  in  part,  we  have  left  the  instructor  in  the  loop  because  ADL  requires 
a  level  of  flexibility  we  cannot  reasonably  program  into  a  machine.  Our  instructors  try 
eveiything  at  their  disposal  to  help  students  learn.  They  try  to  adapt  to  student  needs  and 
to  the  peculiar  demands  of  the  domain.  DC-Legacy  helps  in  this  endeavor  because  it  pro¬ 
vides  a  flexible  but  pedagogically  sound  structure,  multiple  methods  of  instruction,  and  a 
single  gathering  of  "at  the  ready"  resources.  There  are  two  questions  that  come  to  mind 
now.  One  question  is  what  aspects  of  the  domain  were  generally  difficult  or  impossible 
to  remediate.  A  second  question  is  whether  certain  conceptualizations  facilitate  the  stu¬ 
dents'  subsequent  learning. 

Discussion 

In  this  section,  we  described  a  theory  that  can  help  evaluate  misconceptions  in  the 
context  of  instruction.  To  this  end  we  proposed  a  dynamic  assessment  approach  to  as¬ 
sessing  domain  leamability.  In  this  approach,  researchers  try  their  best  to  teach  students. 
Those  concepts  that  students  still  have  difficulty  with  tell  us  something  about  the  compo¬ 
nents  of  the  domain  that  are  particularly  difficult  to  learn,  at  least  with  respect  to  the  in¬ 
struction  that  we  can  provide.  The  results  help  focus  attention  on  those  concepts  that  are 
particularly  problematic,  rather  than  simply  making  a  list  of  possible  misconceptions  in  a 
domain.  Ideally,  the  important  concepts  are  also  identified  with  respect  to  their  impact  on 
future  learning  in  the  domain. 

We  described  several  classes  of  learning  difficulties  (misconceptions)  that  others 
and  we  have  found  for  the  domain  of  electricity.  We  tried  to  organize  these  misconcep¬ 
tions  according  to  the  way  they  fit  into  basic  cognitive  processes  --  differentiation,  sim¬ 
plifying  assumptions,  local  reasoning,  and  the  need  for  framing.  Our  underlying  assump¬ 
tion  is  that  domain  leamability  is  best  understood  as  the  interaction  of  individuals'  cogni¬ 
tive  tendencies,  the  demands  of  the  domain,  and  instruction.  We  constructed  a  DC- 
Legacy  that  targeted  these  different  classes  of  learning  difficulties  to  fulfill  the  instmc- 
tional  component  of  our  assessment.  Using  DC-Legacy,  several  members  of  our  group 
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were  able  to  add  their  own  expertise  to  make  a  rich  dynamic  assessment  environment  for 
learning  DC  circuits. 

We  conducted  a  small  study  to  see  if  there  is  merit  to  the  approach.  The  prelimi¬ 
nary  results  are  promising.  We  found  that  some  commonly  cited  misconceptions,  like  the 
difficulty  of  handling  parallel  resistors,  are  not  problematic  by  the  time  students  leave  a 
typical  college  course  in  electricity.  We  found  that  other  misconceptions,  like  local  rea¬ 
soning  about  the  movement  of  current  from  point  to  point,  are  not  treated  by  our  courses. 
However,  they  are  easily  remediated  and  do  not  have  to  serve  as  blocks  to  learning  the 
domain.  And,  we  found  that  some  aspects  of  the  domain  are  difficult  to  learn  even  with 
special  attention.  In  particular,  it  appeared  that  people  have  trouble  integrating  multiple 
causes  and  this  is  exacerbated  by  faulty  intuitions  that  cause  them  to  focus  on  singular 
causes.  We  suspect  that  most  instruction  does  not  sufficiently  help  students  construct 
mental  models  that  incorporate  both  the  empirical  reasoning  of  causal  intuition  and  the 
helpful  structure  of  mathematics  (Schwartz  &  Moore,  1998).  In  electricity  it  seems  par¬ 
ticularly  important  to  help  students  make  sense  of  the  mathematical  formulas  (qualita¬ 
tively  or  quantitatively)  so  they  may  overcome  the  tendency  towards  minimum  causality. 
In  our  protocols  with  college  professors  and  field  experts  alike,  we  have  found  that  when 
they  come  to  an  obstacle  in  their  reasoning,  they  resort  to  equations  to  solve  difficult  con¬ 
ceptual  problems.  And,  in  our  discussions  of  AC  circuits,  electrical  engineering  experts 
rely  so  heavily  on  mathematics  that  they  often  cannot  even  generate  physical  analogies. 
They  are  reasoning  about  representations,  primarily  mathematical;  the  empirical  phenom¬ 
ena  are  far  in  the  background. 

The  current  theorizing,  the  computer  environment  that  implements  our  theories, 
and  the  empirical  results  present  our  beginning  efforts  at  creating  dynamic  assessment 
tools  that  can  inform  instruction  in  complex  domains.  As  such,  none  of  the  tliree  are 
ideal  and  more  work  is  left  to  be  done. 

5.  INDUCTOR:  DESCRIPTION  AND  PILOT  STUDIES 

Inductor  was  designed  as  an  online  assessment  tool  by  Jay  Pfaffman  to  provide  “a 
means  to  author,  administer,  grade,  and  learn  from  multiple-choice  tests”  (Pfaffman, 
Schwartz,  &  Martin,  2001).  These  tests  are  often  criticized  for  being  shallow  and  for 
promoting  memorization,  but  they  can  be  transformed  into  assessments  for  learning 
(Bransford,  Brown,  &  Cocking,  1999).  They  may  be  redesigned  to  specifically  target 
misconceptions  and  important  knowledge  principles,  and  when  they  are  presented  in  a 
computer  enviromnent,  they  can  be  designed  to  provide  immediate  and  elaborate  feed¬ 
back  to  students  (Hunt  &  Minstrell,  1994).  By  using  an  online  system,  we  may  also  pro¬ 
vide  access  to  outside  resources  for  learning,  such  as  instructional  web  sites  and  simula¬ 
tions  or  animations  (see  Appendix  B  for  an  annotated  listing  of  some  of  the  resources 
used  in  our  tests  with  Inductor).  This  creates  a  learning  environment  that  goes  beyond 
the  typical  sequestered  problem  solving  context  for  taking  tests  in  which  there  is  no  ac¬ 
cess  to  tools  or  resources  (Bransford  &  Schwartz,  1999). 

Inductor  is  implemented  as  a  web-based  tool,  tlirough  the  use  of  PHP  scripting  to 
create  a  web  browser  interface  that  is  connected  to  a  MYSQL  database  backend.  This 
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allows  instructors  with  a  web  browser  to  remotely  create  and  author  test  questions,  and 
students  to  work  on  assignments  outside  of  the  classroom  environment.  Test  authors  may 
include  graphics,  audio,  and  video  with  each  question,  and  provide  general  and  specific 
feedback  for  each  multiple-choice  answer  (both  the  right  and  wrong  ones)  to  a  question. 

Appendix  A  visually  demonstrates  how  a  student  uses  the  online  Inductor  tool 
(see  also:  http://relax.ltc.vanderbilt.edu/onr/demo.php  ).  The  student  selects  a  test  to  take, 
and  is  presented  with  detailed  instructions.  Once  they  are  done  with  the  instructions  and 
the  preliminary  screens,  the  student  sees  a  grid  on  Inductor  depicting  all  of  the  questions 
in  the  test,  organized  by  topic  or  some  other  structure  that  the  designer  has  chosen.  For 
example,  in  Inductor  we  organized  the  test  into  four  categories  of  questions:  DC  resistive 
circuits,  AC  resistive  circuits,  DC  circuits  with  capacitors,  and  AC  RC  (filter)  circuits. 
These  were  laid  out  row-wise.  In  addition,  we  identified  three  classes  of  problems:  diag¬ 
nosis,  design,  and  analysis  problems.  An  icon  associated  with  a  question  indicated  the 
type  of  question  to  the  student.  Inductor  is  set  up  to  provide  continual  feedback  to  stu¬ 
dents  on  their  perfonnance  via  the  “control  your  own  destiny”  (CYOD)  grid  interface. 
Students  access  a  question  by  clicking  on  the  question’s  icon.  They  may  work  on  one 
question  at  a  time,  but  after  answering  a  question  may  revisit  it  any  number  of  times  to 
review  its  contents.  After  answering  a  question,  the  student  returns  to  the  grid.  At  all 
times,  color-coding  on  the  question  icons  (green  for  correct  answers,  red  for  wrong  an¬ 
swers)  tells  the  student  how  well  he  or  she  has  perfonned  thus  far.  At  this  point,  students 
may  review  these  previously  answered  questions,  review  resources,  or  move  on  to  a  new 
question. 

All  questions  are  described  in  text  with  an  accompanying  schematic  or  circuit 
diagram,  and  a  set  of  multiple  choice  answers.  The  student  is  asked  to  pick  the  right  an¬ 
swer  to  the  question,  and  provide  additional  information,  such  as  explanations  in  addi¬ 
tional  boxes  that  we  provide  on  the  screen.  Every  screen  also  has  pointers  to  generic  re¬ 
sources,  as  well  as  specific  ones  that  may  contain  material  relevant  to  that  particular 
question.  In  this  study,  we  required  students  to  develop  a  problem  solving  methodology 
using  invariants.  Invariants  come  in  two  forms.  First,  they  may  pertain  to  laws  of  the  do¬ 
main  that  directly  apply  to  the  problem  solution  being  sought.  Second,  they  are  also 
linked  to  the  set  of  variables  in  the  particular  problem-solving  situation  that  do  not 
change.  Foe  example,  if  one  is  dealing  with  a  flashlight  problem,  where  the  bulb  is  re¬ 
place  by  a  higher  wattage  bulb,  once  can  reason  that  the  source  voltage  (battery)  does  not 
change,  but  the  resistance  of  the  bulb  does.  Therefore,  the  source  voltage  is  an  invariant. 
Depending  on  the  question  asked,  one  can  then  invoke  an  invariant  law,  such  as  Ohm’s 
Law,  the  Power  laws,  or  Kirchoffs  Voltage  Law  to  answer  the  question.  The  list  of  in¬ 
variant  laws  we  have  used  in  the  study  can  be  accessed  at 
http://relax.ltc.vanderbilt.edu/onr/invariants/invariants.html. 

To  promote  and  teach  expert-like  reasoning,  we  made  students  select  the  invari¬ 
ants  they  felt  were  relevant  to  solving  the  given  circuit  problem.  Another  unique  charac¬ 
teristic  of  our  questions  was  that  there  were  no  numbers  associated  with  the  circuit  pa¬ 
rameters  ad  variables.  Students  were  required  to  derive  their  answers  using  qualitative 
reasoning  techniques  (i.e.,  current  will  be  higher,  the  resistor  must  be  shorted,  etc.).  This 
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was  to  get  them  to  reason  from  first  principles  as  opposed  to  putting  in  a  lot  of  effort  into 
numeric  calculations.  After  answering  a  question,  selecting  the  relevant  invariants,  and 
explaining  their  answer,  students  received  feedback  about  their  answer.  If  incorrect,  stu¬ 
dents  were  shown  what  invariants  an  expert  chose  as  relevant  to  the  problem.  In  addi¬ 
tion,  students  answering  incorrectly  received  a  hint  and  links  to  relevant  outside  resources 
to  explore.  Finally,  regardless  of  the  correctness  of  the  answer  to  a  question,  students  re¬ 
ceived  an  expert  explanation  for  the  circuit  problem,  emphasizing  the  invariant  principles 
involved. 

In  a  study  that  used  related  techniques,  Leonard,  Dufresne,  and  Mestre  (1996)  had 
physics  students  describe  the  principles  involved  in  physics  problems  and  write  a  justifi¬ 
cation  for  their  answer.  The  professors  also  described  problem  solving  strategies  during 
their  lectures,  much  like  the  invariant-based  explanations  and  techniques  for  problem 
solving  that  we  present  through  Inductor.  They  found  that  the  students  who  were  taught 
problem  solving  strategies  generated  more  correct  answers  to  problems,  were  less- 
dependent  on  surface  features  of  problems  alone  for  selecting  important  principles  in¬ 
volved  in  solving  problems,  and  were  better  in  recalling  the  major  principles  covered  in 
the  course  months  later. 

There  are  two  major  differences,  however,  between  the  Leonard,  et  al.  study  and 
the  pilot  studies  we  have  run  to  date.  One  is  that  their  study  was  conducted  as  a  semester 
long  course,  and  students  were  graded  on  their  answers,  their  use  of  strategies,  and  their 
identification  of  principles  they  applied  to  solving  problems.  We  have  not  yet  integrated 
Inductor  into  an  electrical  engineering  or  physics  courses,  and  thus  student  participation 
in  these  studies  has  been  voluntary,  and  not  directly  connected  to  their  course  work. 
Also,  their  performance  has  had  no  direct  impact  on  their  class  grades,  and,  as  a  result,  a 
number  of  students  have  not  put  any  significant  effort  in  using  or  learning  from  our  sys¬ 
tem.  This  has  affected  their  motivation  for  participation  and  learning  in  the  tasks.  Sec¬ 
ondly,  the  instructors  in  the  Leonard,  et  al.  study  carefully  reviewed  all  the  students’  writ¬ 
ing  and  provided  feedback  to  the  students  on  their  strategy  use.  This  provided  invaluable 
feedback  and  learning  opportunities,  but  also  undoubtedly  represents  a  significant  in¬ 
vestment  of  time  and  effort  on  the  part  of  the  instructors.  Inductor  makes  a  trade-off  by 
providing  automatic  feedback  in  the  form  of  hints  and  expert  explanations  and  learning 
resources  to  students.  Our  focus  is  more  on  self-assessment,  and  we  hope  students  will 
be  more  motivated  to  learn  by  conventional  instruction  or  through  on  line  systems,  once 
they  realize  what  their  particular  deficiencies  in  understanding  the  material  are. 


First  Pilot  Test  of  Inductor  (November,  2000) 

We  ran  a  first  pilot  study  of  the  Inductor  tool  using  our  DC  and  AC  test  questions 
in  November  2000.  We  created  two  16-item  tests  that  involved  simple  DC,  simple  AC, 
DC  capacitor,  and  AC  RC  circuits.  We  wrote  expert  explanations  for  each  of  the  circuit 
problems  that  emphasized  the  invariant  properties  of  the  circuit.  These  were  presented  to 
each  student  after  answering  each  question.  We  found  and  organized  outside  resources, 
such  as  web  sites  and  Java  simulations  that  we  felt  were  valuable  for  learning  the  specific 
invariant  principles  our  test  questions  targeted  (see  Appendix  B). 
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In  order  to  explore  learning  effects  with  the  Inductor  tool,  we  counter-balanced 
the  order  in  which  students  took  both  of  our  tests,  to  provide  some  measure  of  control  for 
any  differences  in  difficulty  between  the  two  tests.  Our  goal  was  to  find  out  how  the  stu¬ 
dents  fared,  with  their  answers,  selection  of  invariants,  and  explanations  as  they  pro¬ 
gressed  through  Inductor.  After  completing  both  tests,  we  asked  students  to  fill  out  a  fol¬ 
low-up  survey  to  get  their  feedback  on  the  tool  and  on  the  use  of  invariants. 

Methods 

Participants.  Forty  undergraduate  students  in  a  first  year  electrical  engineering  course 
volunteered  to  take  part  in  the  study. 

Procedure  and  Design.  Students  used  Inductor  with  the  procedure  described  in  Appendix 
A.  Two  16-item  test  assignments  (tests  A  &  B)  were  given.  Both  tests  had  questions  in¬ 
volving  simple  DC  circuits,  AC  circuits,  DC  capacitor  circuits,  and  AC  RC  circuits.  The 
questions  on  tests  A  and  B  were  different  but  similar  in  content. 

The  design  for  this  initial  study  was  for  half  of  the  students  to  take  test  A  first  and 
then  test  B,  and  the  other  half  to  take  test  B  and  then  A,  counterbalancing  any  differences 
in  difficulty  between  the  items  on  both  tests.  If  students  learnt  while  answering  the  ques¬ 
tions  and  receiving  feedback  and  resources,  then  performance  should  improve  over  time. 

We  recorded  the  answer  each  student  selected  for  each  multiple-choice  question, 
the  invariant  principles  they  selected  as  having  relevance  to  the  problem,  and  any  initial 
explanations  for  their  answer,  as  well  as  revisions  to  their  explanations  after  having  re¬ 
ceived  feedback.  After  completing  both  tests,  students  filled  out  the  follow-up  survey. 

Results  of  First  Pilot  Study 

Sixteen  of  the  forty  students  completed  all  questions  on  both  of  the  tests.  We  attributed 
the  lower  participation  rate  to  the  fact  that  the  tests  were  not  required  or  connected  to 
their  current  electrical  engineering  course,  the  length  of  the  tests,  and  the  approaching 
holidays  and  finals  in  late  November  and  early  December.  Five  of  the  students  who  fin¬ 
ished  both  tests  were  in  the  group  that  took  test  A  first  and  then  test  B  second  (“Class 
A”),  and  eleven  students  took  test  B  first  and  test  A  second  (“Class  B”). 

Test  Scores.  An  analysis  of  the  total  scores  indicated  that  Test  B  was  significantly  easier 
than  Test  A  (F(l,14)=6.95,  p=.02).  Test  B  averaged  66.8%  correct,  and  Test  A  60.1%. 
This  difference  was  true  for  all  categories  of  questions  of  the  tests  (DC,  AC,  DCC, 
ACRC)  except  for  AC.  Class  A  averaged  55%  correct  answers  and  Class  B  averaged 
67%  correct,  however  this  was  not  a  significant  difference  (F(l,14)=2.09,  p=.171),  due  to 
the  low  number  of  students  in  Class  A  (5)  and  the  high  variance  between  individual 
scores.  The  interaction  between  class  and  test,  graphed  below  in  Fig.  3,  is  also  non¬ 
significant  (F(l,14)=2.83,  p=.l  15).  Four  of  five  students  in  Class  A  improved  from  their 
first  test  to  the  second,  but  only  three  of  eleven  in  Class  B  did.  Class  B  participants  first 
took  the  easier  test  (B)  and  then  the  harder  one.  We  do  not  have  data  such  as  pre-test 
scores  or  student  GPAs  to  control  for  individual  and  test  differences  to  better  reveal 
learning  effects  in  the  student  scores  using  Inductor. 
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Percent  Correct  Answers  by  Class  and 
Order  of  Test  Taken 


jo  1st  Test 
■  2nd  Test 


Class  A  Class  B 

Class  (Group) 


Figure  3:  Interactions  between  class  and  test 

Surprisingly,  we  found  students  answered  more  correctly  for  AC  circuit  questions 
than  DC  (F(l,14)=6.95,  p=.023).  Students  also  had  difficulties  with  capacitor  questions 
(F(l,14)=22.85,  p<.0001).  The  three  graphs  in  Fig.  4  reveal  test  scores  by  category  of 
question  and  class. 

Background  for  Invariant  Selection  Analysis.  For  each  circuit  problem,  we  used  tliree  ex¬ 
perts  to  select  the  invariants  that  were  most  relevant  and  helpful  for  solving  the  problem. 
After  initial  selection  they  met  to  come  to  a  consensus  about  the  relevant  invariants  for 
each  of  the  questions.  They  also  identified  the  invariants  that  were  clearly  irrelevant.  The 
impedance  of  an  inductor,  for  example,  is  irrelevant  for  a  RC  circuit.  The  remaining  in¬ 
variants  were  placed  in  a  third  category.  A  link  could  be  drawn  between  these  invariants 
and  the  circuit  behavior,  but  they  were  not  particularly  useful  for  solving  the  problem 
asked  about  the  circuit.  Ohm’s  law,  for  example,  applies  to  all  circuits  with  resistive 
components,  but  it  is  not  always  necessary  for  a  particular  question  about  a  circuit. 

We  needed  a  way  to  quantify  how  well  students  selected  invariants  for  a  particu¬ 
lar  problem.  In  our  case,  a  simple  percent  correct  measure  is  insufficient  for  characteriz¬ 
ing  students’  use  of  invariants,  because  it  rewards  students  who  select  more  invariants 
regardless  of  their  importance  to  the  problem.  To  control  for  such  response  biases,  we 
utilized  a  nonparametric  discrimination  measure  known  as  Yule ’s  Q.  Nelson  (1984)  con¬ 
trasted  simple  percentage  correct  measures,  d’  measures  and  Yule’s  Q,  and  advocated 
Yule’s  Q  over  d’  on  the  basis  that  it  was  thought  to  make  weaker  assumptions  about  the 
data  and  required  fewer  observations.  For  our  purposes  this  measure  rewards  the  selec¬ 
tion  of  invariants  that  our  experts  agreed  were  appropriate  for  the  problem,  while  control¬ 
ling  for  the  selection  of  clearly  irrelevant  invariants. 

The  Yule’s  Q  measure  was  constructed  first  by  calculating  the  percentage  of  in¬ 
variants  a  student  chose  out  of  those  invariants  experts  chose  as  relevant  (h,  or  hit  rate)  as 
well  as  the  percentage  of  invariants  a  student  chose  out  of  those  invariants  experts 
deemed  clearly  irrelevant  (f,  or  false  alarm  rate).  Invariants  that  are  technically  correct 
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but  less  relevant  to  a  problem  were  ignored  in  this  computation.  The  Yule’s  Q  score  was 

h-f 


then  calculated  by  the  formula: 


.  A  Q  of  one  implies  perfect  discrimination 


h-2Jh  +  f 

of  the  relevant  from  irrelevant  invariants,  and  zero  implies  pure  chance  performance. 


Figure  4:  Test  scores  by  category  for  Class  A  and  Class  B 


Results  for  Invariant  Selection  and  Discrimination.  The  average  discrimination  of  invari¬ 
ants  (as  calculated  by  Yule’s  Q)  when  a  correct  answer  to  a  circuit  problem  was  chosen 
was  0.53.  The  average  discrimination  when  an  incorrect  answer  was  chosen  was  0.39. 
Thus  students  were  more  likely  to  select  those  invariants  that  experts  deemed  relevant  on 
questions  they  answered  correctly. 
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However,  we  found  that  students’  selection  of  relevant  invariants  declined  from 
the  beginning  of  the  tests  to  the  end.  The  graphs  in  Fig.  5  reveal  this  pattern  across  each 
of  the  categories  of  questions  and  across  both  classes.  Different  explanations  may  be 
provided  for  this  pattern  of  results.  One  is  that  the  questions  grow  more  difficult  from  the 
beginning  to  the  end  of  a  test.  Another  explanation  is  a  fatigue  or  indifference  factor. 
Students  may  have  been  concentrating  only  on  getting  the  correct  answer  to  questions, 
and  gradually  paid  less  attention  to  the  invariant  selection.  In  this  study  students  actually 
selected  the  answer  to  the  question  first  and  then  selected  the  invariants.  In  the  second 
study,  we  changed  this  and  also  asked  students  to  only  select  the  primary  invariant  rele¬ 
vant  to  a  circuit  problem,  not  all  the  invariants  that  are  relevant  at  all. 


Class  B  Selection  of  Invariants 
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Figure  5:  Selection  of  Invariants  across  questions 

Student  Survey  Responses.  Appendix  C  summarizes  and  counts  the  type  of  student  re¬ 
sponses  to  questions  on  a  follow-up  survey.  In  general,  students  liked  the  practical  appli¬ 
cability  of  the  concepts  and  liked  the  outside  resources  and  hints  we  provided.  Some  of 
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the  students  would  have  liked  to  see  even  more  resources.  However,  some  students  felt 
the  wording  of  some  of  the  questions  and  explanations  were  confusing.  We  worked  on 
rewriting  and  improving  our  test  questions  and  expert  explanations  for  the  second  study, 
and  indeed,  no  students  mentioned  wording  as  a  problem  in  the  follow-up  survey  given 
after  the  second  test  study. 

When  asked  what  invariants  are,  students  mentioned  they  were  like  rules  or  laws, 
and  unchangeable  properties  of  a  circuit.  They  felt  that  considering  the  invariants  that  are 
involved  in  a  circuit  gave  them  a  “baseline”  or  place  to  start  solving  a  problem  with  a  cir¬ 
cuit. 

Second  Pilot  Test  of  Inductor  (April  2001) 

Before  testing  the  Inductor  tool  a  second  time,  we  rewrote  the  questions  and  sim¬ 
plified  the  expert  explanations  for  the  circuit  problems.  We  reduced  the  total  number  of 
questions  and  redistributed  them  to  create  two  more  evenly  balanced  and  equivalent  tests. 
We  changed  the  order  of  the  procedure  for  answering  questions  so  that  the  student  was 
first  required  to  think  of  and  select  the  primary  invariant  he  or  she  felt  was  the  key  to  de¬ 
riving  the  solution.  After  invariant  selection,  the  student  chose  an  answer  and  provided  an 
explanation,  just  like  the  first  test.  The  list  of  invariants  from  which  to  choose  was  rear¬ 
ranged  and  reworded  to  be  simpler  and  less  redundant. 

In  this  study,  one  of  our  goals  was  to  remove  some  of  the  glitches  of  the  first  ex¬ 
periment,  viz.,  ambiguous  such  as  the  wording  of  the  questions,  unwieldy  expert  explana¬ 
tions,  and  unbalanced  tests.  We  also  directed  our  study  toward  determining  the  influence 
of  invariants  on  problem  solving.  To  create  two  conditions,  we  randomly  assigned  half  of 
the  volunteer  participants  to  a  condition  in  which  they  did  not  receive  the  online  instruc¬ 
tion  on  invariants,  and  they  were  not  prompted  to  select  invariants  for  each  question. 

Some  of  the  other  issues  affecting  the  prior  study  remained,  however.  We  had  not 
yet  attempted  to  integrate  use  of  Inductor  into  an  electrical  engineering  course,  and  there¬ 
fore,  participation  in  the  study  was  still  voluntary  and  not  directly  comiected  to  students’ 
current  coursework.  Also  we  had  not  yet  integrated  our  own  resources  into  Inductor  such 
as  simulations  and  online  videos.  The  resources  available  in  Inductor  still  consisted  of 
those  outside  web  sites  we  selected  as  valuable  for  learning  the  concepts  presented  in  our 
tests. 

Methods 

Participants.  Twenty  undergraduate  students  in  a  second  year  electrical  engineering 
course  volunteered  to  participate  online  and  take  the  tests. 

Procedure  and  Design.  All  students  completed  two  14-item  tests  with  the  online  Inductor 
tool.  One  group  of  students  in  this  study  did  not  receive  instruction  about  the  use  of  in¬ 
variants  and  did  not  have  to  select  invariants  after  reading  each  question.  The  invariants 
group  did  receive  instruction  about  the  use  of  invariants  for  solving  problems  before  be- 
gimiing  the  tests.  They  also  were  asked  to  select  the  primary  invariant  property  of  the 
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circuit  for  each  problem.  Besides  this,  the  two  groups  received  identical  instruction  and 
feedback.  Members  of  both  groups  were  also  pointed  to  resources  in  the  form  of  outside 
web  links,  and  hints  on  possible  conceptual  errors  they  may  have  made  when  they  missed 
a  question.  After  answering  correctly  or  revising  one’s  explanation  for  the  answer  to  a 
question,  the  Inductor  tool  presented  an  expert  explanation  for  the  problem. 

The  two  tests  were  constructed  to  be  as  similar  as  possible  with  respect  to  content 
and  difficulty.  Each  consisted  of  four  basic  DC  circuit  questions,  four  basic  AC  circuit 
questions,  two  DC  capacitor  circuit  questions,  and  four  questions  concerning  AC  cireuits 
with  capacitors  (RC,  or  filter  circuits).  In  the  previous  study  we  have  found  that  students 
have  more  difficulty  with  questions  involving  capacitors. 

Results  of  Second  Pilot  Study 

Six  of  the  twenty  participants  completed  both  of  the  tests.  We  believe  the  low 
participation  rate  was  partly  due  to  the  fact  that  use  of  the  tool  was  not  connected  to  their 
current  class  work.  In  the  future,  we  will  be  working  with  professors  to  make  Inductor  a 
supplemental  resource  for  classes.  Also,  the  Inductor  tool  was  described  to  students  only 
as  an  online  test.  In  future  versions.  Inductor  will  incorporate  more  instructional  re¬ 
sources,  and  we  can  better  characterize  it  as  an  online  learning  environment  rather  than 
an  online  test  system. 

Two  of  the  six  participants  did  not  receive  invariants-based  instruction  and 
prompts,  and  four  did  receive  instruction  on  the  use  of  invariants. 

Test  Scores.  Overall,  the  participating  students  scored  an  average  61%  correct  answers 
on  the  first  test,  and  82%  correct  on  the  second  test,  an  improvement  of  21%.  Five  of  the 
six  participants  had  improved  perfonnance  from  the  first  to  the  second  test.  There  were 
no  significant  differences  between  the  scores  of  the  invariants  and  no-invariants  groups, 
but  two  of  the  invariants  group  members  scored  perfectly  on  the  second  test  (see  Table 
1).  One  student  (ID  6)  had  a  lot  of  difficulty  answering  the  questions  correctly  and  did 
not  show  improvement,  but  all  students  were  asked  to  explain  their  reasoning  behind  the 
problems,  and  thus  these  explanations  reveal  more  concerning  their  answers. 


1  (No  Invariants)  57% 

79% 

2  (No  Invariants)  71% 

93% 

3  (Invariants) 

50% 

86% 

4  (Invariants) 

71% 

100% 

5  (Invariants) 

71% 

100% 

6  (Invariants) 

43% 

36% 

Table  1:  Percentage  Correct  Answers  on  Each  Test  for  Each  Participant 

As  in  earlier  studies,  we  found  that  students  had  the  most  difficulty  with  problems 
dealing  with  capacitors.  However,  in  this  study,  student  performance  in  the  second  test 


23 


was  higher  in  all  categories,  and  the  largest  improvement  was  in  the  DC  capacitor  prob¬ 
lems  (Fig.  6). 


Figure  6:  Percentage  Correct  Answers  in  Each  Category  for  First  and  Second  Tests. 

Student  Explanations.  After  reading  a  question  and  selecting  an  answer,  students  were 
asked  to  provide  an  explanation  for  their  answer.  If  their  answer  was  incorrect,  after  re¬ 
ceiving  hints  and  links  to  outside  resources,  students  were  asked  to  revise  their  explana¬ 
tion  for  the  problem.  Finally,  regardless  of  the  correctness  of  their  initial  answer,  the  In¬ 
ductor  tool  presented  an  expert  explanation  for  the  circuit  problem,  and  students  were 
asked  for  final  comments.  Thus  students  could  generate  three  sets  of  comments  for  each 
question. 

The  students  who  did  not  receive  invariant-based  instruction  and  prompts  typi¬ 
cally  wrote  shorter  and  less  precise  explanations  than  the  group  that  received  invariants 
instruction.  Many  of  these  students  seemed  to  use  the  outside  resources  we  provided. 
For  example,  two  students  gave  explanations  for  AC  capacitor  questions  resembling  the 
infomiation  provided  from  the  resources:  "at  high  frequencies,  it  will  act  like  a  >  short” 
and  "at  infinity  =  short.  ”  Both  groups  of  students  also  initially  revealed  misconceptions 
{"higher  resistance  means  more  power  is  absorbed,  ”  "internal  resistance  rises,  ”  "the  in¬ 
ternal  resistance  is  lower  for  low  frequencies,  ”  "John's  battery  will  be  required  to  work 
harder  to  push  current  through  the  larger  resistor”),  errors  {"after  long  time,  all  voltage 
will  be  across  capacitor,  ”  "since  they  are  in  series  the  voltage  should  be  the  same  across 
all  of  the  bulbs  ”),  and  admitted  to  making  some  guesses.  However,  the  responses  of  stu¬ 
dents  who  received  invariants  prompts  and  instruction  revealed  they  more  often  at¬ 
tempted  to  revise  and  correct  their  misconceptions: 

>  Example  1: 

>  D'  explanation;  "Since  frequency  and  current  are  related  linearly,  and  in¬ 
crease  in  frequency  will  increase  current.  ” 

>  2"^  explanation:  "Actually,  current  and  frequency  have  no  relation  so  cur¬ 
rent  does  not  change  with  frequency.  ” 
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>  Example  2: 

>  1st  explanation:  "More  current  flows  through  the  bulbs  when  they  are  in 
series,  this  makes  a  brighter  light.  ” 

>  2nd  explanation:  "More  voltage  through  the  bulbs  in  parallel  will  make  for 
brighter  bulbs. " 

'P-  3rd  explanation:  "Power  is  what  determines  the  light  intensity.  More 
power  makes  for  more  intensity.  ” 


The  invariants  group  also  articulated  some  of  the  invariant  principles  they 
learned: 

>  "There  is  only  one  path  for  the  current  to  flow  through  and  both  bulbs  lie  on  this 
path.  ” 

>  "With  low  frequencies  most  of  the  voltage  is  across  the  capacitor  since  its  imped¬ 
ance  is  high  when  frequency  is  low.  ’’ 

P  "Because  the  lower  resistance  in  Peter's  circuit  will  result  in  more  power  con¬ 
sumption.  lower  resistance  =  more  power” 

P  "Since  it  is  in  series  connection,  the  same  resistances  and  current  will  produce  the 

same  power  for  all  the  bulb,  p  =  i^R.” 

P  "When  you  go  from  parallel  to  series,  the  intensity  will  decrease  since  the  voltage 
is  split  across  both  bulb  1  and  bulb  2.  ” 

P  "Because  the  lower  resistance  will  need  more  power. " 
"the  lower  resistance  will  result  in  more  power  being  dissipated  because 


The  non-invariants  group  demonstrated  some  knowledge  of  invariants  too,  how¬ 
ever: 

P  "The  bulbs  are  in  series.  Therefore,  the  current  through  them  must  be  the  same.  ” 
P  "R  doesn  't  depend  on  frequency.  " 

P  "The  impedance  of  a  capacitor  changes  inversely  with  frequency. " 

P  "KVL.” 


We  believe  the  outside  resources  may  have  helped  participants  in  the  non¬ 
invariants  group  induce  some  of  the  invariant  principles.  The  outside  resources  were  se¬ 
lected  because  they  specifically  target  concepts  related  to  the  invariants.  This  may  also 
partly  explain  why  the  non-invariant  group  perfonned  as  well  as  the  invariants  group  in 
correctly  answering  the  questions,  and  improved  as  much  on  the  second  test  as  the  invari¬ 
ants  group.  The  questions  themselves  targeted  important  invariant  principles,  and  were 
also  more  clearly  written  than  in  the  first  pilot  study  (no  students  mentioned  problems 
with  wording  in  the  follow-up  survey  for  the  second  pilot  study). 


Student  Survey  Responses.  After  completing  both  tests,  students  answered  questions  on 
a  follow-up  survey.  The  students  responded  that  they  liked  and  used  the  outside  re¬ 
sources  and  the  hints  we  provided  after  answering  incorrectly.  One  student  even  sug¬ 
gested,  “/  was  veiy  impressed  with  the  information  provided  to  learn  from  mistakes.  I 
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think  that  that  information  should  be  provided  regardless  of  whether  the  answer  was 
right  or  wrong  so  that  if  the  answer  was  just  a  guess  I  could  solidify  my  understanding.  ” 

Students  also  mentioned  they  thought  the  test  questions  helped  reinforce  concepts 
they  had  learned,  and  also  helped  them  to  better  apply  what  they  had  learned.  When  we 
asked  those  participants  who  received  instruction  on  invariants  what  they  believed  invari¬ 
ants  are,  and  how  they  are  used,  the  students  primarily  thought  of  invariants  as  a  method 
for  solving  a  problem.  When  asked  what  is  an  invariant,  one  student  responded:  "A  cir¬ 
cuit  invariant  is  a  certain  method  of  circuit  analysis  needed  to  solve  the  problems  pre- 
sented.(ex:  ohms  law,  power,  etc).  ”  When  asked  why  it  is  useful  to  analyze  a  circuit  by 
considering  the  invariants,  that  student  also  responded:  "It  gives  the  student  an  idea  of 
where  to  start  the  problem.  ”  Other  students  also  responded:  "It  lets  you  know  how  to 
solve  the  circuit  and  how  to  solve  like  circuits  in  the  future,  ”  and  "It  allows  one  to  find 
and  use  the  necessary  method  of  solution  quicker.  ” 

6.  SUMMARY  AND  FUTURE  WORK 

The  complex  and  invisible  nature  of  the  electricity  domain  makes  learning  and 
understanding  of  electricity  concepts  difficult.  Since  the  domain  is  so  challenging,  and  it 
is  not  possible  to  “see”  or  “feel”  current  or  voltage,  it  is  difficult  for  students  to  develop 
proper  conceptions  about  basic  electricity  concepts.  In  the  domain  of  DC,  we  had  dis¬ 
covered  a  variety  of  misconceptions  about  current,  voltage,  resistance  and  power 
(Biswas,  et  al,  1997).  For  example,  we  had  found  that  people  have  problems  differentiat¬ 
ing  voltage  from  current,  so  they  talk  about  voltage  "flowing"  through  the  circuit  or  bat¬ 
tery  being  a  source  of  current.  They  do  not  understand  that  the  potential  difference  or  the 
voltage  across  the  battery  causes  the  current  to  flow  through  the  wires.  Often,  they  think 
that  current  flows  sequentially,  like  water  through  an  empty  pipe  (so,  the  first  bulb  in  a 
row  of  two  bulbs  would  light  first).  Or,  that  the  current  in  a  circuit  is  constant  at  all  times 
(and  that  is  a  property  of  the  battery),  that  a  resistor  "slows  down"  the  current  so  that  the 
flow  actually  alleviates  right  "after"  the  resistor,  and  so  on. 

Our  more  recent  studies  show  that  similar  misconceptions  appear  in  students’  un¬ 
derstanding  of  the  basic  concepts  of  the  AC  domain.  Many  DC  misconceptions  seem  to 
carry  over  to  the  AC  domain.  In  addition,  the  time-varying  nature  of  voltage  and  current, 
and  the  fact  they  go  from  positive  to  negative  and  back  confuses  students  because  they 
think  that  current  cannot  change  directions  and  still  flow  in  the  circuit.  Therefore,  they 
rationalize  that  the  sine  wave  form  represents  a  spatial  rather  than  temporal  property  of 
current,  and  that  current  travels  as  a  spatial  “wave”  through  the  wires.  The  key  to  under¬ 
standing  AC  phenomena  can  be  described  in  two  general  points:  (i)  to  understand  the  no¬ 
tions  of  time  vaiying  voltage  and  current,  and  (ii)  the  implications  of  their  time  varying 
nature  in  analyzing  circuit  behavior. 

We  also  described  our  approach  to  remediating  students  lack  of  knowledge  and 
misunderstanding  by  developing  our  own  dynamic  assessment  approach  that  takes  into 
account  not  only  the  learning  potential  of  individuals,  but  the  leamability  of  the  domain 
concepts  as  well.  The  results  of  a  preliminary  study  to  test  the  expectations  we  had  of  our 
approach  are  promising.  We  found  that  some  commonly  cited  misconceptions,  (like  the 


26 


difficulty  of  handling  parallel  resistors)  are  not  existent  by  the  time  students  move  to 
more  advanced  courses  in  electricity.  We  found  that  other  misconceptions,  like  local  rea¬ 
soning  about  the  movement  of  current  from  point  to  point,  can  be  easily  remediated  and 
do  not  have  to  serve  as  obstacles  to  further  learning.  We  also  found  that  some  aspects  of 
the  domain  are  difficult  to  learn  even  with  special  attention. 

More  recently,  we  have  focused  on  Inductor,  a  system  for  self-assessment  and 
learning  problem  solving  methods  that  are  modeled  after  expert  reasoning.  We  have  in¬ 
troduced  the  notion  of  invariants  to  guide  the  problem  solving  process,  and  have  devel¬ 
oped  a  web-based  multi-choice  test  that  helps  students  realize  their  lack  of  understanding 
of  key  domain  concepts.  The  make  up  of  the  multiple-choice  test  was  guided  by  the  pro¬ 
tocol  studies  that  we  had  conducted  earlier.  Preliminary  studies  demonstrate  the  effec¬ 
tiveness  of  the  Inductor  system  in  helping  student’s  self-assess  their  difficulties.  Students 
have  also  provided  positive  feedback  on  the  resources  we  provide,  and  the  general  ap¬ 
proach  to  problem  solving  with  invariants. 

Our  future  work  will  include  the  further  development  of  our  ADL  framework  by 
introducing  challenge  problems  that  are  directly  linked  to  tasks  that  Naval  technicians 
will  be  involved  in.  The  challenge  is  to  develop  instructional  resources  that  help  student 
teclmicians  better  understand  basic  AC  phenomena,  and  their  implications  on  maintaining 
and  trouble  shooting  complex  equipment.  We  hope  that  by  focusing  on  real  world  de¬ 
vices  and  applications,  students  will  develop  better  intuitions  about  the  key  characteristics 
of  the  phenomena,  enhancing  their  ability  to  learn  and  apply  the  concepts  in  a  variety  of 
real  world  situations. 
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A  student  begins  by  connecting  to  the  Inductor  website  and  selecting  a  username 
for  him  or  herself  (see:  http://relax.ltc.vanderbilt.edu/onr/demo.php  ).  After  typing  in  a 
username,  the  student  clicks  “Sign  Me  Up!”  and  then  clicks  the  login  link.  The  assigned 
tests  are  displayed  along  with  progress  information  and  due  dates. 

Clicking  “Take  Test”  takes  the  student  to  a  page  of  instructions.  These  instructions  in- 
fonn  the  student  about  the  nature  of  the  test  questions,  and  explain  the  use  of  invariants 
for  answering  the  questions.  Recently  we  have  also  added  a  video  by  Dr.  Bhuva  explain¬ 
ing  what  invariants  are  and  how  to  use  them  for  thinking  about  a  problem. 


Test  Instructions 

Thankj^ou  for  participating  in  this  e?peiiniental  test  for  understanding  how  people  leam  and  reason 
about  electricitj^prohleniiS. 

The  foIlo\\1ng  test  has  multple  choice  questions  about  basic  DC  circuits,  AC  circuits,  including 
circuits  vith  and  without  capacitors.  None  of  the  questions  requires  a  calculator  or  quantitathre 
ans^rers. 

On  the  next  page  }uuuill  see  links  to  all  the  questions  in  a  4  jxfw  matrix.  The  questions  on  the  top 
roAvs  are  gener^y  easier  than  the  questions  on  the  loAver  roAVS,  but  you  may  complete  them  in  any 
order  you  Amh- 


Yoiu'  Record 


ifesf  name 

assignedteslkey 

Start  Time  |  ComjtletiDn  Time  j 

Due  Date  j 

jTestB . ; 

Talr,e  Test 

(None)  1 

(None)  i 

Jim4th  1 

ITestA 

Talie  Test 

Jim  5tli  08:18 

(None) 

Jun4th  j 
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Test  Matrix  Page 

After  clicking  the  “Take  Test”  link  at  the  bottom  of  the  instructions  page,  students  are 
taken  to  the  matrix  of  questions  for  the  test.  These  are  organized  by  category,  and  the 
icons  for  each  question  may  be  used  to  provide  additional  information  as  well.  The  stu¬ 
dent  may  click  any  of  the  icons  to  attempt  a  question. 

Taking  Test 


1.DC 

Circuits 

DC 

drcuits 

DC 

circuits 

: 

; 

DC 

drcuits 

; 

DC 

drcuits 

i 

2.  x4C 
Circuits 
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Sample  Question  Page 

Below  is  a  picture  of  a  sample  test  question.  First  there  is  a  question  along  with  a  picture. 
Next  the  students  are  asked  to  choose  the  invariants  they  feel  are  important  for  solving 
the  problem.  Each  invariant  in  the  list  is  a  link  that  pops  up  a  new  window  with  more 
infonnation  and  resources  relevant  to  the  invariants.  Next,  students  select  their  answer 
and  give  an  explanation  for  why  they  chose  the  invariants  and  answer. 

Two  identical  bulbs  that  were  originally  connected  in  parallel  to  a  DC  source  are  now  conirected  to  the  same 
source  in  series. 

Will  tire  total  light  intensity  of  the  tV'ti  bulbs  talwn  together  change? 


VH^iich  of  these  irwariants  do  jrou  feel  are  most  important  for  tlds  problem? 
n  a.  Ohm's  Law 
D  b.  Impedance  of  a  Capacitor 
D  c.  Chanre  heldbv  a  Capacitor 
D  d.  Impedance  of  an  Inductor 
n  e.  Inductor  arid  Flm: 

□  f  ■  pQi.yer 

D  g.  Kirchoffs  Laws  of  Consers^tion 
LJ  h.  Eflectr.»'e  Resigtaiure 

Select  your  answer  for  the  question: 

A .  O  The  light  intensity  v/ill  not  chai^ . 

B .  O  The  light  intensit jr  ^?n!^  increase . 

C .  O  Tiie  light  intensity  ^-ill  decrease . 

D.  U  We  need  to  Imow  the  \'alue  of  the  resistance  of  tlie  bulbs  to  compute  tlriis  ans^^^r. 

Please  e:q>laiji%vliy  you  chose  the  mirariaiit(s)  and  answer: 


32 


Correct  Answer  Feedback  Page 

In  this  example,  the  answer  was  correct.  The  student  is  still  given  an  expert  explanation 
for  the  answer. 

Thajite  for  your  zjisweT, 


Two  identical  bulbs  that  were  originall3r  connected  in  parallel  to  a  DC  source  are  now  connected  to  the  same 
source  in  series. 

Will  the  total  hght  intensity  of  the  Xwo  bulbs  tal^Len  together  change? 


.  Tlie  light  intensity  mR  not  change . 

B .  The  light  intensity  vdU  increase . 

C.  The  l^ht  decrease.  (Your  answer) 

D.  "We  i^eed  to  luiow  the  value  of  the  resistance  of  the  bulbs  to  compute  this  ansu-^r. 

Your  ans^ver  ^^'as  correct! 

Yom'  invaiiants :  Expert-chosen  invariants : 

f.  Power  f.  Power 

g.  KiichofTs  of  Conserv^ation  h.  EfTectr.^  Flesistance 


Here  are  some  hints  for  sohdng  this  problem: 

Tlie  total  resistance  of  the  i\vo  bulbs  for  tlie  series  circuit  (R1+E2)  is  greater  titan  that  for  the  two  bulbs  in  the 
paiiillel  configuration  ( R1*R2/(R1+R2) ).  The  power  delivered  by  the  battery  can  be  computed  as  P=V^2/R, 
where  P.  is  tlie  effecth^  resistance  of  the  circuit  .Tlierefore,  since  the  voltage  V  does  not  clmge,  tlie  pov-^r 
delivered  in  tlie  parallel  case  is  greater. 


P.eady  foi  another  questkm. 


Incorrect  Answer  Example 

For  this  next  sample  question,  an  incorrect  answer  is  entered. 


An  e:q:iensive  AC  to  DC  power  supply  coir^rter  is  connected  to  an  AC  pou-^r  source.  A  decision  lias  to  be 
made  on  wliere  to  place  fuse(s)  to  protect  the  server  electronics  on  the  DC  side  from  AC  pov;^r  surges. 


V^riuch  of  these  inv'ai'krLts  do  you  feel  are  most  irapoilant  for  tliis  problem? 
n  a.  Ohirhs  Lw 
D  b.  Impedaiice  of  a  Capacitor 
n  c.  Charge  held  by  a  Capacitor 
n  d.  Impedance  of  art  Inductor 
n  e.  Induictor  aiid  Flux: 

0  f  Power 

n  g.  Kiii:hofrs  of  Conser/ation 
di  h.  Effecth'^e  Resistance 

Select  yoxu  answer  for  the  question: 

A.  Circuit  1  is  not  properly  fused.  It  W'tU  only  partially  protect  the  serv’er  electronics 

B .  Circuit  2  is  not  properly  fused.  It  will  only  partially  protect  tlie  ser^'er  electronics 

C.  Circuit  3  is  the  onl3r  circuit  that  is  properly  fused. 

D.  O  It  does  rot  matter  where  jrou  put  the  fuse  in  tlie  circuit. 

Please  e^^lain  why  you  chose  the  i]n'aiiaiit(s)  and  ansu'er: 

It  is  better  to  have  2  fuses  to  catch  positive  and  negative  ~ 
surges. 


Submit  Your  Answer  and  Explanation 
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Incon  ect  Answer  Feedback 

The  feedback  in  this  example  does  not  tell  a  student  the  correct  answer  yet.  First,  student 
is  shown  the  invariants  an  expert  chose  for  reasoning  about  this  problem.  A  hint  is  also 
given  with  links  to  helpful  resources,  which  in  the  future  may  also  include  simulations  or 
videos.  The  student  enters  a  revised  explanation  based  on  this  feedback. 

x'i.  Circuit  1  is  not  properl3r  fused.  It  ^^oll  only  partially  protect  the  semr  electronics. 

B.  Circuit  2  is  not  properly  fused.  It  will  onljr  partially  protect  tire  ser^^er  electronics. 

C .  Circuit  3  is  the  only  circuit  that  is  properly  fused!  (Your  ansu'er) 

D.  It  does  not  matter  rvlrere  jmu  put  the  fuse  in  the  circuit. 

Your  ans^r'er  to  the  question  u'as  incorrect 

Your  mvariants :  Eoqrert-chosen  ircv’aiiants : 

f  Poi,ver  g.  KirchofTs  Laws  of  Coriservation 


Here  are  some  hints  for  sohdng  this  problem: 

Horv  do  jjrou  apphsT  KirchofTs  current  kw'  to  derh/e  tire  answer? 

•  KircIiofTs  Voltage  Law  tutorial 

•  KirchofTs  Current  Law  tutorial 

•  BiitchofT  arid  his  laucs 

Ho’.v  would  you  revise  your  eiipkriation? 


I  guess  one  fuse  is  enough  because  the  current  at  any  point 
in  the  circuit  is  always  the  same. 


Submit  Your  Answer  and  Explanation 


Finally,  the  student  is  told  the  correct  answer,  and  an  expert  explanation  for  the  problem 
is  given.  This  expert  explanation  is  given  at  the  end  of  correctly  and  incorrectly  an¬ 
swered  questions. 

The  correct  ansivei  is;  It  does  not  rtiatter  where  3rou  put  tlie  fuse  in  tlie  circuit. 

Here  are  some  hints  for  sohdng  this  prohlem: 

I'Vccordirig  to  KCL,  the  total  current  entering  a  node  is  equal  to  the  total  current  leaving  the  node .  Hence  current 
at  each  point  in  the  circuit  willbe  the  same  in  this  case.  So  to  protect  the  pov'er  supply,  we  need  only  to  bieal: 
the  current  path  wiien  power  surges  occur,  and  it  does  not  matter  where  we  do  it.  So  choice  (d)  is  tiie  correct 
ansv'er. 
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Reviewing  a  Question  Previously  Answered 

After  finishing  answering  a  question,  the  student  is  taken  back  to  the  matrix  of  questions. 
Previously  answered  questions  are  surrounded  with  a  green  (for  correct)  or  red  (for  incor¬ 
rect)  box  and  have  a  link  for  reviewing  the  question  and  one’s  answers  and  explanations. 


The  question  wos:  An  ejq^ensivie  AC  to  DC  po'wier  suppl3r  coitvierter  is  connected  to  an  AC  pou^r  souice.  A 
decision  has  to  be  made  on  where  to  place  fuse(s)  to  protect  the  server  electronics  on  the  DC  side  from  AC  pou-'er 
surges.  E}:amine  the  three  circuit  digrams  bebw  and  decide  wliich  describes  the  proper  placement  of  the  fuse. 

Circuit  1  is  not  properljr  fused.  It  will  onl3r  partia]l3r  protect  the  servier  electronics. 

B.  Circuit  2  is  not  properl3r  fused.  It  wall  onl3r  partia]l3i'  protect  the  server  electronics. 

C.  Circuit  3  is  the  only  circuit  that  is  properly  fuse£  (Your  answer) 

D .  !f  does  not  matter  you  put  the^fuse  in  the  circuit.  ( CORRECT) 

Explanntioiis 

Turn  off  my  ex}:iLanatiQns 

First  e:q?lanation:  It  is  better  to  have  2  fuses  to  catch  positr^T  and  negatr^  surges. 

Second  ejplanation:  I  guess  one  fuse  is  enough  because  the  current  at  an3r  point  in  the  circuit  is  alw/a3i^  the  same. 
Tldrd  e:qrlanation: 


E.a;:k  to  Questions 


APPENDIX  B 

Outside  Resources  Provided  With  Inductor 
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In  order  to  foster  further  learning  during  the  course  of  using  Inductor,  outside  re¬ 
sources  are  provided  for  students  to  explore,  in  addition  to  the  hints  and  invariant-based 
explanations  given  within  Inductor.  These  outside  resources  consisted  of  websites  with 
tutorials,  explanations,  and  simulations  of  DC  and  AC  circuit  concepts.  Links  to  these 
resources  are  provided  in  two  contexts.  On  the  page  listing  descriptions  of  all  the  invari¬ 
ants,  links  to  outside  resources  are  listed  next  to  the  invariant  to  which  they  apply  (see 
http://relax.ltc.vanderbilt.edu/onr/invariants/invariants.html  ).  Resources  are  also  pro¬ 
vided  for  students  to  explore  after  first  answering  a  question  incorrectly.  Inductor  gives 
helpful  hints  for  thinking  about  the  problem,  including  a  list  of  outside  resources  targeted 
towards  the  invariants  involved  in  the  problem.  Below  is  an  annotated  list  of  some  of  the 
resources. 

RLC  Filter  Circuits: 

a)  http://users.erols.com/renau/impedance.html  (java  applet) 

b)  http://webphvsics.ph.msstate.edu/jc/librarv/21-5/index.html  (java  applet) 

c)  http://anthrax.phvsics.indiana.edU/~dzierba/Honors2/Schedule/Week9/AC/capacitor.h 
tml 

d)  http://anthrax.phvsics.indiana.edu/~dzierba/Honors2/Schedule/Week9/AC/lowhigh.ht 
ml 

Resources  a  and  b  are  Java  simulations  of  an  RLC  AC  filter  circuit.  The  simula¬ 
tions  are  interactive,  allowing  a  student  to  manipulate  the  values  of  the  voltage,  fre¬ 
quency,  capacitance,  inductance,  and  resistance  and  dynamically  update  the  graphs  of  the 
circuit  behavior.  The  circuit  behavior  is  graphed  in  both  the  frequency  and  time  domains. 
The  advantage  of  these  features  is  that  students  can  explore  and  test  the  circuit’s  invariant 
properties,  such  as  that  the  impedance  of  the  capacitor  (Xc)  decreases  with  frequency. 
Resources  c  and  d  additionally  provide  the  student  with  two  simple  fundamental  ideas  to 
help  their  reasoning  about  these  circuits:  a  capacitor  approaches  an  open  circuit  at  low 
frequencies  and  a  short  circuit  at  high  frequencies,  and  the  difference  between  a  low-pass 
and  high-pass  filter. 

RC  Circuits: 

a)  http://www.phvs.hawaii.edu/~teb/iava/ntnujava/rc/rc.html  (java  applet) 

b)  http://www.sweethaven.com/acee/fon'ns/frm0802.htm 

c)  http  ://www.  sweethaven.  com/acee/fomis/frm  1 003  .htm 

Students  often  have  difficulty  understanding  the  behavior  of  capacitors  in  DC  and 
AC  circuits.  Resource  a  is  an  illustrative  simulation  of  a  DC  circuit  charging  and  dis¬ 
charging.  It  dynamically  graphs  the  output  voltage  over  time,  and  metaphorically  illus¬ 
trates  current  flow  in  the  wires  in  a  manner  consistent  with  Kirchoff  s  current  law.  Re¬ 
sources  b  and  c  also  explain  the  invariant  behavior  of  a  capacitor  in  a  DC  circuit  and  link 
the  behaviors  shown  in  the  simulation  to  mathematical  formulations. 
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Kirchoff  s  Laws  of  Conservation: 

a)  http://www;phvsics.uogueIph.ca/tutorials/ohin/Q.ohm.KVL.html  (including  j ava) 

b)  http://www.phvsics.uoguelph.ca/tutorials/ohm/Q.ohm.KCL.html 

c)  http://www.ece.utexas.edu/~adulev/lab/index.htnil 

Resources  a  and  b  very  simply  state  the  basic  rules  of  Kirchoff  s  voltage  and  cur¬ 
rent  laws  and  how  voltage  and  current  are  conserved  in  a  circuit.  These  are  two  of  the 
most  fundamental  invariant  relationships  governing  circuit  behavior.  Resource  c  addi¬ 
tionally  provides  some  history  behind  these  principles  and  additional  circuit  examples. 

Future  Resources.  We  are  currently  developing  our  own  resources  that  more  specifically 
illustrate  the  invariant  relationships  in  electrical  circuits.  For  example,  an  interactive 
graphing  tool  can  illustrate  how  the  impedance  of  a  capacitor  changes  with  the  frequency 
of  an  AC  source,  and  how  that  affects  output  current  and  voltage.  A  Java-based  circuit 
simulation  tool  will  show  a  circuit  from  different  conceptual  views.  One  illustrates  the 
topological  relationships  in  a  circuit,  providing  a  diagram  of  the  circuit  along  with  con¬ 
trols  for  changing  various  properties  such  as  voltage,  frequency,  resistance,  and  capaci¬ 
tance.  A  second,  connected  view  graphs  the  properties  of  the  circuit  by  time  or  fre¬ 
quency.  An  expert  view  will  provide  text  or  movies  explaining  what  to  notice  about  the 
circuit  and  the  invariant  properties  of  the  circuit  behavior.  Additional  views  might  illus¬ 
trate  systems  that  are  analogical  to  the  circuit  and  its  behavior,  or  describe  the  mathe¬ 
matical  relationships  that  characterize  the  circuit. 
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APPENDIX  C 

Summary  of  16  Responses  to  the  Follow-Up  Survey  Given  After  the 
First  Test  Study  (December,  2000) 

1)  Do  you  think  having  more  questions  like  the  ones  you  answered  would  make 
a  useful  addition  to  a  course?  Please  give  us  a  few  positive  reasons  and  a  few 
negative  reasons. 

•  7/16  liked  seeing  the  practical  applicability  of  the  concepts 

•  4/16  referred  to  using  or  reinforcing  knowledge  they  have  previously  learned 

•  2/16  thought  it  was  too  time  consuming  or  not  time  efficient 

•  6/16  had  problems  with  the  wording  and  choice  of  questions  themselves 

2)  If  you  can,  please  write  down  the  different  classes  of  problems  that  you  were 
asked  to  solve. 

•  12/16  mentioned  AC  and  DC  circuits  and  components  (RLC) 

•  7/16  mentioned  one  or  all  the  categories  design,  troubleshooting,  analysis 

3)  Were  there  features  of  the  on-line  assessment  system  that  you  found  useful  or 
interesting?  Please  give  us  a  few  positive  features  and  a  few  negative  features. 

•  8/1 6  liked  the  hints  or  explanations 

•  2/16  mentioned  liking  typing  in  their  own  explanations 

•  7/16  mentioned  problems  with  the  questions  or  wording 

4)  Did  you  learn  anything  about  your  strengths  and  weaknesses  in  the  AC  do¬ 
main  by  completing  this  assessment? 

•  16/16  learned  something  about  their  understanding  of  AC  circuits. 

•  See  the  individual  responses  below,  some  are  very  telling. 

5)  What  is  a  circuit  invariant? 

•  5/16  mentioned  components  (3  of  whom  only  mentioned  components) 

•  8/16  mentioned  rule/law/property 

•  6/16  mentioned  "unchangeable"  or  "constant" 

6)  Can  you  think  of  any  reasons  that  it  is  useful  to  begin  an  analysis  of  a  circuit 
by  considering  the  invariants? 

•  9/16  mentioned  it  gives  you  a  foundation/baseline/"place  to  start 
with'Voutline/picture 

•  10/16  mentioned  it  makes  it  easier  to  begin  solving  the  problem,  choosing  the 
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equations  to  use 

7)  How  useful  did  you  find  some  of  the  outside  resources  in  the  test,  and  what 
kinds  of  resources  would  you  find  most  helpful  to  add? 

•  10/16  found  them  helpful  or  useful 

•  3/1 6  thought  some  were  wordy,  overwhelming,  or  too  simple 

•  5/16  would  like  more  resources/pictures  added  or  improvements  to  some  existing 
resources 

8)  Did  the  questions,  explanations,  and  resources  provided  in  the  first  test  make 
it  easier  for  you  to  derive  the  answers  in  the  second  test  you  took. 

•  13/16  said  yes 

•  5/16  said  it  clued  them  onto  the  fonnat  of  the  problems  &  questions,  knew  what 
to  expect,  questions  were  similar 

•  2/16  said  it  made  them  pay  more  attention,  learn  from  mistakes 


